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(54) Image display system, image processing method, and information storage medium 



(57) An image display system, image processing 
method, and program that make it possible to reproduce 
the colors of an image more rapidly in accordance with 
the user's preferences. The image display system in- 
cludes:- a color gamut calculation section 1 60 that cal- 
culates a target color gamut based on a target profile 
within a target profile storage section 162 that was se- 
lected by the user, and also calculates a displayable 
color gamut based on a projector profile within a projec- 



tor profile storage section 164 and environmental infor- 
mation from a colored-light sensor 60 that detects the 
visual environment; a matrix generation section 1 22 that 
generates a conversion matrix according to the relation- 
ship between the target color gamut and the displayable 
color gamut; and a matrix converter section 124 that us- 
es the thus-generated conversion matrix to convert im- 
age information; whereby image information is convert- 
ed and image is displayed. 
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Description 

[0001] Japanese Patent No. 2000-344822. filed on 
November 13, 2000. and Japanese Patent No. 
2000-296027, filed on September 27 : 20C1 , are herein 5 
incorporated by reference in their entirety. 

BACKGROUND OF THE INVENTION 

Field of the Invention 10 

[0002] The present invention relates to an image dis- 
play system, image processing method, and information 
storage medium. 

75 

Description of Related Art 

[0003] Various color conversion systems such as the 
color managemenl syslem (CMS) have been proposed 
in order to provide uniformity in an image view. 2° 
[0004] However, the desired image view can differ, 
depending on individuals and location. 
[0005] As one example, the standardized method for 
displaying images in Japan is NTSC, whereas one of 
the standardized methods for displaying images in Eu- 25 
rope is PAL. 

[0006] A situation that can therefore arise in which an 
image that was generated on the assumption that it 
would be viewed as NTSC in Japan is displayed in Eu- 
rope to Europeans, giving rise to a difference in which 30 
the image is viewed, from consideration of the prefer- 
ences of Europeans. 

[0007] For that reason, it is not sufficient merely to 
convert the image information (such as the RGB sig- 
nals), but it is necessary to convert the image informa- 35 
tion in accordance with an image characteristic such as 
the display method selected by the user. 
[0008] Since an image view is also affected by factors 
such as ambient light, it is also necessary to detect the 
visual environment and take that visual environment into *o 
consideration when converting the image information. 
[0009] During the conversion of image information in 
accordance with the image characteristic and the visual 
environment, the information used for the conversion 
must be generated. However storing conversion infor- 45 
mation for all assumed image characteristics and visual 
environments beforehand in a storage region places 
pressures upon that storage region. 
[0010] In addition, image information that is generat- 
ed in real time must be converted in real time. 50 

SUMMARY OF THE INVENTION 

[0011] The present invention was devised in the light 
of the above-described technical problems and has as 55 
an objective thereof the provision of an image display 
system, image processing method, and information 
storage medium that enable faster reproduction of an 



optimal image view in accordance with an image char- 
acteristic selected by a user. 

(1 ) In order to solve the above described technical 
problems, according to a first aspect of the present 
invention, there is provided an image display sys- 
tem which converts image information used for dis- 
playing an image and displays the image, based on 
environmental information obtained by a visual en- 
vironment detection means which detects visual en- 
vironment in a display area in which the image is 
displayed; the image display system comprising: 

matrix generation means which generates a 
conversion matrix used for conversion, based 
on the environmental information and an image 
characteristic selected by a user, so as to dis- 
play the image that is suited to the visual envi- 
ronment and the image characteristic; 
matrix conversion means which converts the 
image information, based on the generated 
conversion matrix; and 

image display means which displays the image 
based on the converted image information; 

wherein the image characteristic is based on 
at least one of an image display method and an im- 
age classification. 

(2) According to a second aspect of the present in- 
vention, there is provided a computer- readable in- 
formation storage medium which stores a program 
for converting image information used for displaying 
an image, based on environmental information ob- 
tained by a visual environment detection means 
which detects visual environment in a display area 
in which the image is displayed, 

the program causing a computer to function 

as: 

matrix generation means which generates a 
conversion matrix used for conversion, based 
on the environmental information and an image 
characteristic selected by a user, so as to dis- 
play the image that is suited to the visual envi- 
ronment and the image characteristic: and 
matrix conversion means which converts the 
image information, based on the generated 
conversion matrix; and 

wherein the image characteristic is based on 
at least one of an image display method and an im- 
age classification. 

(3) According to a third aspect of the present inven- 
tion, there is provided an image processing method 
which converts image information used for display- 
ing an image, based on environmental information 
obtained by a visual environment detection means 
which detects visual environment in a display area 
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in which the image is displayed, the image process- 
ing method comprising: 

a matrix generation step of generating a con- 
version matrix used for conversion, based on 5 
the environmental information and an image 
characteristic selected by a user so as to dis- 
play the image that is suited to the visual envi- 
ronment and the image characteristic; and 
a matrix conversion step of converting the im- '0 
age information, based on the generated con- 
version matrix. 

The present invention enables conversions that 
are faster than a case in which a look-up table (here- ^ 
inafter abbreviated to LUT) is used as conversion 
information, by generating a conversion matrix for 
use as conversion information and using that con- 
version matrix in the conversion of the image infor- 
mation, which also enables a reduction in the ded- 20 
icated space within a storage region. 

It therefore becomes possible to display an im- 
age that conforms to the image characteristic se- 
lected by the user and the visual environment, in 
real time. 25 

Note that the image display method in this case 
refers to a method such as NTSC, PAL, or SECAM. 
The image classification refers to a system such as 
RGB orsRGB. 

(4) The image display system may further comprise: 30 

a color gamut calculation means which calcu- 
lates a target color gamut based on the image 
characteristic and also calculates a displayable 
color gamut that is displayable by the image 35 
display means in the visual environment, based 
on the environmental information; 

wherein the matrix generation means may 
generate different conversion matrices according to *o 
a relationship between the displayable color gamut 
and the target gamut, the relationship including a 
case where the displayable color gamut is larger 
than the target color gamut, a case where the dis- 
playable color gamut is smaller than the target color 45 
gamut, a case where the displayable color gamut 
matches the target color gamut, and a case where 
the displayable color gamut has a portion that over- 
laps the target color gamut and a portion that does 
not overlap the target color gamut. 50 

(5) The information storage medium which stores a 
program may cause the computer to function as a 
color gamut calculation means which calculates a 
target color gamut based on the image characteris- 
tic and also calculates a displayable color gamut 55 
that is displayable by the image display means in 

the visual environment, based on the environmental 
information; . - 



wherein the matrix generation means may 
generate different conversion matrices according to 
a relationship between the displayable color gamut 
and the target gamut, the relationship including a 
case where the displayable color gamut is larger 
than the target color gamut, a case where the dis- 
playable color gamut is smaller than the target color 
gamut, a case where the displayable color gamut 
matches the target color gamut, and a case where 
the displayable color gamut has a portion that over- 
laps the target color gamut and a portion that does 
not overlap the target color gamut. 

(6) In the image processing method, the matrix gen- 
eration step may include a step of calculating a tar- 
get color gamut based on the image characteristic 
and also calculating a displayable color gamut that 
is displayable by the image display means in the 
visual environment, based on the environmental in- 
formation; and 

the matrix generation step may generate dif- 
ferent conversion matrices according to a relation- 
ship between the displayable color gamut and the 
target gamut, the relationship including a case 
where the displayable color gamut is larger than the 
target color gamut, a case where the displayable 
color gamut is smaller than the target color gamut, 
a case where the displayable color gamut matches 
the target color gamut, and a case where the dis- 
playable color gamut has a portion that overlaps the 
target color gamut and a portion that does not over- 
lap the target color gamut. 

The relationship between a color gamut based 
on an image characteristic and a color gamut that 
can be displayed by the image display means de- 
pends on the visual environment and the image 
characteristic. For that reason, it is not possible to 
reproduce an image appropriately by a method of 
converting image information that uses only a single 
conversion matrix. 

The present invention makes it possible to re- 
produce an image more appropriately by dividing 
the relationships into the above described four pat- 
terns and generating a conversion matrix in corre- 
spondence to each of those patterns. 

(7) In the image display system and the information 
storage medium, the matrix generation means may 
generate the conversion matrix that emphases ei- 
ther hue reproducibility or color gamut reproducibil- 
ity, when the displayable color gamut is smallerthan 
the target color gamut or when the displayable color 
gamut has a portion that overlaps the target color 
gamut and a portion that docs not overlap the target 
color gamut. 

(8) in the image processing method, the matrix gen- 
eration step may generate the conversion matrix 
that emphases either hue reproducibility or color 
gamut reproducibility, when the displayable. color 

..gamut, is. smaller than the target color gamut or 
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when the displayable color gamut has a portion that 
overlaps the target color gamut ™d a portion that 
does not overlap the target co amut. 

This makes it possible to .• oduce an image 
more appropriately: by generating a conversion ma- 5 
trix that emphasizes either hue reproducibility or 
color gamut reproducibility. 

(9) The image display system may further comprise 
a projection-type display device having: 

10 

the color gamut calculation means; 

the matrix generation means; 

the matrix conversion means; 

the image display means; and 

calibration image generation means which gen- ^ 

erates a calibration image: 

wherein the image display means may project 
the generated calibration image for display on the 
display area: and 20 

wherein the visual environment detection 
means may detect the visual environment in the dis- 
play area in which the calibration image is dis- 
played. 

Since this enables generation of a calibration 
image within the projection-type display device, it 
makes it possible for the projection -type display de- 
vice itself to perform calibration, without having to 
input a calibration image from an external input de- 
vice such as a PC to the projection-type display de- 30 
vice. 

(10) The image processing method may further 
comprise the steps of: 

generating a calibration image : prior to com- 35 
pensation of the image information; 
displaying the generated calibration image in 
the display area; and 

detecting the visual environment within the dis- 
play area in which the calibration image is dis- *o 
played : and generating the environmental infor- 
mation. 

[0012] This makes it possible to detect the visual en- 
vironment more appropriately, by using a calibration tm- 45 
age. It is therefore possible to reproduce the image view 
more appropriately. 

BRIEF DESCRIPTION OF THE DRAWINGS 

50 

[0013] 

Fig. 1 is a schematic illustrative view of an image 
display system according to one embodiment of the 
present invention; 55 
Fig. 2A is a schematic diagram of a target color 
gamut and a displayable color gamut when the tar- 
get color gamut and the displayable color gamut 



match : and Fig. 2B is a schematic diagram of a tar- 
get color gamut and a disp. decolor gamut when 
the displayable color gam. arger than the target 
color gamut: 

Fig. 3A is a schematic diagram of a target color 
gamut and a displayable color gamut when the dis- 
playable color gamut is smaller than the target color 
gamut., and Fig. 3B is a schematic diagram of a tar- 
get color gamut and a displayable color gamut when 
the target color gamut has a portion that overlaps 
the displayable color gamut and a portion that does 
not overlap the displayable color gamut: 
Fig. 4A is a schematic diagram of color gamuts after 
color gamut mapping that places priority on color 
gamuts and Fig. 4B is a schematic diagram of color 
gamuts after color gamut mapping that places pri- 
ority on hues; 

Fig. 5 is a functional block diagram of an image 
processor section within a projector in accordance 
with one embodiment of the present invention: 
Fig. 6 is a flowchart of the procedure for image 
processing in accordance with one embodiment of 
the present invention; and 

Fig. 7 is a flowchart of the procedure for matrix gen- 
eration and conversion processing in accordance 
with one embodiment of the present invention. 

DETAILED DESCRIPTION OF THE EMBODIMENTS 

[0014] Embodiments of the present invention applied 
to an image display system that uses a liquid crystal pro- 
jector are described below by way of example, with ref- 
erence to the accompanying figures. Note that the em- 
bodiments described below do not limit the scope of the 
invention defined by the claims laid out herein. Similarly, 
the overall configuration of the embodiments below 
should not be taken as limiting the subject matter de- 
fined by the claims herein. 

Overall System 

[0015] A schematic illustrative view of an image dis- 
play system in accordance with one embodiment of the 
present invention is shown in Fig. 1. 
[0016] A projector 20 : which is one type of a projec- 
tion-type display device provided substantially facing a 
screen 10. projects an image for a predetermined pres- 
entation. A presenter 30 gives a presentation to an au- 
dience, while using a light spot 70 projected from the 
laser pointer 50 to point at a desired position of an image 
in an image display region 1 2, which is a display-receiv- 
ing area on the screen. 

[0017] During such a presentation, the image view on 
the image display region 12 will vary greatly, depending 
on factors such as the type of the screen 1 0 and ambient 
light 80. When the same white is displayed, for example, 
the type of the screen 1 0 could make it seem to be white 
with a yellow cast or white with a blue cast. Similarly, 
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differences in the ambient light 80 could make the same 
white appear to be a bright white or a dull white. 
[001 8] Recently, this projector 20 has become smaller 
and easy to transport. For that reason, it has become 
possible to perform presentations at a client's location : 
by way of example : but it is difficult to adjust colors to 
match the environment at the client's location and the 
manual adjustment of colors at the client's location takes 
too much time. 

[0019] With a prior-art projector color modification is 
based on an input-output profile that indicates input-out- 
put characteristics that are specific to that particular pro- 
jector, so no consideration is paid to the visual environ- 
ment in which the image is projected and displayed. 
Note that this "profile" means "characteristic data". 
[0020] However it is difficult to make the view of 
colors uniform in this manner without taking the visual 
environment into account. The view of colors is deter- 
mined by three factors: light, the reflection or transmis- 
sion of light by objects, and vision. 
[0021] This embodiment of the present invention im- 
plements an image display system that can reproduce 
appropriate colors, by detecting the visual environment 
of light and the reflection or transmission of light by ob- 
jects. 

[0022] If the objective is to reproduce colors appropri- 
ately, there may be differences in the appropriate colors, 
depending on the user or the location at which the colors 
are reproduced. 

[0023] If the projector 20 is to be used in Japan, for 
example, it is considered that the user would want to 
reproduce the colors of the image in accordance with 
the NTSC method, whereas if the projector 20 is to be 
used in Japan, it is considered that the user might want 
to reproduce the colors of the image in accordance with 
the PAL method. 

[0024] In such a situation, it is necessary for the user 
to reproduce the desired image colors, regardless of the 
location in which the projector 20 is used. 
[0025] With this embodiment of the invention, the pro- 
jector 20 is configured in such a manner that the colors 
of images can be adjusted on the basis of a selection of 
the user's image display method or the like. 
[0026] More specifically, the device is provided with a 
colored-light sensor 60 that functions as visual environ- 
ment detection means for detecting the visual environ- 
ment, as shown in Fig. 1 . and environmental information 
from the colored-light sensor 60 is input to the projector 
20. The colored-light sensor 60 measures environmen- 
tal information (more specifically, RGB or XYZ tristimu- 
lus values) within the image display region 12 of the 
screen 10. 

[0027] The projector 20 is provided with a conversion 
means for generating a matrix used for conversion, 
based on selection information such as environmental 
information from the colored-light sensor 60 and the us- 
ers image display method, then using that conversion 
matrix for convening image information to be used for 



the image display. 

[0028] An image display system that enables appro- 
priate image color reproduction can be implemented by 
detecting the visual environment on the basis of envi- 
5 ronmental information and determining the user prefer- 
ences from selection information. 
[0029] In addition, this embodiment calculates the dis- 
playable color gamut that can be displayed by the pro- 
jector 20 in the visual environment in which the presen- 
ce tation is given, and also uses an image display method 
selected by the user to calculate a target color gamut. 
Image processing is done by comparing the thus-ob- 
tained displayabie color gamut and target color gamut, 
so that the colors that are as close as possible to the 
15 target color gamut can be displayed by the projector 20. 

Color Gamuts 

[0030] A schematic diagram of a target color gamut 
20 and a displayabie color gamut when the target color 
gamut and the displayabie color gamut match is shown 
in Fig. 2A and a schematic diagram of a target color 
gamut and a displayabie color gamut when the display- 
able color gamut is larger than the target color gamut is 
25 shown in Fig. 2B. Similarly, a schematic diagram of a 
target color gamut and a displayabie color gamut when 
the displayabie color gamut is smaller than the target 
color gamut is shown in Fig. 3A and a schematic dia- 
gram of a target color gamut and a displayabie color 
30 gamut when the target color gamut has a portion that 
overlaps the displayabie color gamut and a portion that 
does not overlap the displayabie color gamut is shown 
in Fig. 3B. 

[0031] In Figs. 2A to 3B, the solid lines denote target 
35 color gamuts and the broken lines denote displayabie 
color gamuts. In addition, the intersection of lines from 
each vertex of each triangular color gamut towards the 
central portion of the triangular shape is the white spot. 
[0032] Note that the color gamuts in Figs. 2A to 3B 
40 are defined in the plans as color coordinates (x, y). In 
this case, x = X/ (X + Y + Z) and y = Y/(X + Y + Z). Each 
of X : Y, and Z is a stimulus value in the XYZ color display 
system. 

[0033] Since there are two variable factors (the image 
45 characteristic and the visual environment), the relation- 
ship between the target color gamut and the displayabie 
color gamut is not fixed, it can be broadly classified into 
the four patterns shown in Figs. 2A to 3B. 
[0034] The method of converting image information 
so differs somewhat depending on which of these four pat- 
terns is appropriate. For example, if the displayabie 
color gamut covers the whole of the target color gamut, 
as shown in Figs. 2A and 2B : the target image can be 
reproduced appropriately by using an ordinary conver- 
55 sion method. 

[0035] If, however the displayabie color gamut does 
not cover the entire target color gamut, as shown in Figs. 
. 3A.and 3B, it will not be possible.to reproduce the target 
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image appropriately by using an ordinary conversion 
method. 

[0036] In such a case ; it is necessary to perform color 
gamut mapping (this could also be called color gamut 
compression) to allocate colors of the target color gamut 
that are outside the dispiayable color gamut to colors 
within the target color gamut. 

[0037] This embodiment of the present invention uses 
either a method that places priority on color gamuts or 
a method that places priority on hues ; as the color gamut 
mapping method. 

[0038] A schematic diagram of color gamuts (map- 
ping color gamuts) after color gamut mapping that plac- 
es priority on color gamuts is shown in Fig. 4A and a 
schematic diagram of color gamuts after color gamut 
mapping that places priority on hues is shown in Fig. 4B. 
[0039] In Figs. 4A and 4B : broken lines denote dis- 
piayable color gamuts and dot-dot-dash lines denote 
target color gamuts. Figs. 4A and 4Bshow examples in 
which the target color gamut and dispiayable color gam- 
ut of Fig. 3B overlap partially. 

[0040] In Fig. 4A, for example, a vertex D of the target 
color gamut is within the dispiayable color gamut ABC 
but a vortex E and a vertex F of the target color gamut 
is on the outside of the dispiayable color gamut ABC. It 
is therefore not possible to reproduce colors in the vi- 
cinity of the vertex E and the vertex F. 
[0041 ] If it is required to display such colors that can- 
not be reproduced in this case, color gamut mapping is 
performed to ensure that those colors are reproduced 
as close as possible. 

[0042] With this embodiment, the color gamut map- 
ping places priority on either color gamut or hue. 
[0043] If the priority is on color gamut, for example, a 
point H that is close as possible to the vertex E and a 
point I that is close as possible to the vertex F are ob- 
tained from intersections between the triangular shape 
DEF and the triangular shape ABC. Note that since the 
vertex D is within the triangular shape ABC, it can be 
used without modification as a vertex G of the new color 
gamut. 

[0044] The triangular shape GH I obtained in this man- 
ner is the mapping color gamut obtained when priority 
is on color gamut, in other words, that obtained when 
consideration is placed on making the mapping color 
gamut as extensive as possible. 
[0045] II the priority is on hue, for example, points K 
and L where lines from the vertices of the triangular 
shape DEF towards the white spot intersect the edges 
of the triangular shape ABC. Note that since the vertex 
D is within the triangular shape ABC , it can be used with- 
out modification as a vertex J of the new color gamut. 
[0046] The triangular shape JKL obtained in this man- 
ner is the mapping color gamut obtained when priority 
is on hue, in other words, that obtained when consider- 
ation is placed on reproducing the hues as accurately 
as possible. Color has three attributed: brightness., enro- 
-ma: and hue.-Of these three, human eyes are most sen- 



sitive to hue. It is therefore possible to use the projector 
20 to reproduce colors that are closer to the target color 
gamut, by obtaining a mapping color gamut that places 
priority on hue. 

5 [0047] The mapping color gamuts of Figs. 2A and 2B 
can use the target color gamut without modification. 
[0048] This embodiment of the invention generates a 
conversion matrix for the conversion of image informa-. 
tion that enables the reproduction of the mapping color 

10 gamut as described above, then used the thus-gener- 
ated conversion matrix to convert the image information . 

Functional Blocks 

15 [0049] The description now turns to the functional 
blocks of the image processor section of the projector 
20 that comprises this matrix generation means. 
[0050] A functional block diagram of an image proc- 
essor section 1 00 within the projector 20 in accordance 

20 with one embodiment of the invention is shown in Fig. 5. 
[0051] The projector 20 inputs an R1 signal, a G1 sig- 
nal, and a B1 signal (which form an RGB signal in analog 
form, sent from a PC or the like) to an A/D converter 
section 110, and uses the projector image processor 

25 section 1 00 to perform color conversion on an R2 signal, 
a G2 signal, and a B2 signal which have been converted 
into digital form by a CPU 200. 

[0052] An R3 signal, a G3 signal, and a B3 signal that 
have been subjected to the color conversion are input 

30 to a D/A converter section 180. and an R4 signal, a G4 
signal, and a B4 signal that have been converted into 
analog form are input to a light valve (L7V) drive section 
1 90 that is part of the image display means, to drive liq- 
uid crystal light bulbs to display an image. 

35 [0053] The projector image processor section 100 
comprises a projector color converter section 120, a 
color gamut calculation section 160, and a calibration 
signal generation section 150. 

[0054] The color gamut calculation section 1 60 com- 
40 prises a target profile storage section 1 62 and a projec- 
tor profile storage section 164. 
[0055] The calibration signal generation section 150 
generates calibration image signals. These calibration 
' image signals are input to the projector color converter 
45 section 1 20 as a digital-format R2 signal, G2 signal, and 
B2 signal, in a similar manner to the signals output from 
the A/D converter section 110. 

[0056] Since the calibration image signals are gener- 
ated within the projector 20 in this manner, calibration 

50 can be done by the projector 20 itself, without having to 
input calibration image signals to the projector 20 from 
an external input device such as a PC. 
[0057] The projector color converter section 1 20 ref- 
erences a projector profile that is managed by the pro- 

55 jector profile storage section 1 64 to convert the RGB sig- 
nals (R2 signal, G2 signal, and B2 signal) from the cal- 
ibration signal generation section 150 into digital RGB 
signals (R3 signal, G3 signaL.and B3 signal) suitable for 
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projector output. 

[0058] The projector color converter section 120 com- 
prises a matrix generation section 1 22, which generates 
a conversion matrix for the conversion of the digital sig- 
nals (R2 signaL G2 signal and B2 signal) that are image 
information, and a matrix converter section 124. which 
uses the thus-generated conversion matrix to convert 
the image information. 

[0059] More specifically, the matrix generation sec- 
tion 122 generates a matrix for use in conversion ena- 
bling reproduction of the mapping color gamut calculat- 
ed by the color gamut calculation section 160. 
[0060] The description now turns to the color gamut 
calculation section 160. 

[0061] The color gamut calculation section 160 com- 
prises the target profile storage section 1 62 and the pro- 
jector profile storage section 164. More specifically the 
color gamut calculation section 1 60 calculates the map- 
ping color gamut that was described with reference to 
Figs. 2 to 4 : in such a manner as to obtain the preferred 
colors selected by the user and also the view of image 
colors that conforms to the visual environment, based 
on a target profile selected by the user, environmental 
information from the colored-light sensor 60, and a pro- 
jector profile. 

[0062] Note that the "target profile" in this case is a 
type of input-output characteristic data for the colors that 
will be the target. More specifically, each target profile 
is data that defines correspondences between factors 
such as RGB luminance signals and tristimulus values 
(X. Y Z) ; by way of example. With this embodiment of 
the invention, each target profile is implemented by a 
matrix that converts RGB luminance signals into tristim- 
ulus values (X, Y, Z). A plurality of profiles corresponding 
to a plurality of image characteristics that can be select- 
ed by the user is provided as target profiles. 
[0063] A "projector profile" is a type of input-output 
characteristic data corresponding to the type of the pro- 
jector 20. More specifically, each projector profile is data 
that defines correspondences between factors such as 
RGB luminance signals and tristimulus values (X, Y Z) 
obtained in practice when an image is displayed in an 
ideal environment on the basis of those RGB luminance 
signals, by way of example. With this embodiment of the 
invention, each projector profile is implemented by a 
matrix that converts RGB luminance signals into tristim- 
ulus values (X, Y Z). 

Image Processing Flow 

[0064] The description now turns to the flow of image 
processing using the above described components, 
with reference to flowcharts. 

[0065] A flowchart of the procedure for image, 
processing in accordance with one embodiment of the 
invention is shown in Fig. 6. 

[0066] First of all, before the presentation, the user of 
the projector 20 selects one image characteristic from 



a plurality of image characteristics allocated to operating 
buttons of the projector 20. More specifically, the outer 
surface of the projector 20 is provided with selection but- 
tons for image characteristics such as NTSC. PAL. or 

5 SEC AM, and the user is induced to press one of those 
selection buttons to select one image characteristic. 
[0067] This selection information is sent to the projec- 
tor image processor section 100. The projector image 
processor section 100 turns on the flag of the thus-se- 

10 lected target profile, from a plurality of target profiles of 
the target profile storage section 1 62, based on that se- 
lection information. 

[0068] In this manner the projector image processor 
section 100 selects the target profile corresponding to 

15 the user's selection (step S2). 

[0069] After the target profile corresponding to the us- 
er's selection has been selected, the projector 20 gen- 
erates calibration signals (R2, G2, B2) from the calibra- 
tion signal generation section 150. 

20 [0070] The calibration signal generation section 150 
outputs those calibration signals to the projector color 
converter section 120. 

[0071] The projector color converter section 120 uses 
a default (initial state) conversion matrix to convert the 
25 calibration signals and outputs them as digital RGB sig- 
nals (R3, G3, B3). 

[0072] The D/A converter section 180 converts the 
digital RGB signals into analog RGB signals (R4, G4, 
B4). The L/V drive section 1 90 drives liquid crystal light 
30 valves, based on the analog RGB signals (R4, G4, B4). 
The projector 20 projects a calibration image to display 
on the image display region 12 (step S4). 
[0073] In the state in which the calibration image is 
displayed on the image display region 12, the colored- 
35 light sensor 60 detects tristimulus values for detecting 
the visual environment (step S6). 
[0074] In this manner, the visual environment can be 
detected more appropriately, by using the calibration im- 
age in the detection of the visual environment. It is there- 
to fore possible to reproduce the image view more appro- 
priately. 

[0075] The projector color converter section ^gen- 
erates a conversion matrix based on the thus-detected 
visual environment, then uses that conversion matrix to 
^5 convert the image information (step S8). 

[0076] This matrix generation and conversion 
processing (step S8) will now be described more spe- 
cifically. 

[0077] A flowchart of the procedure of matrix genera- 
50 tion and conversion processing in accordance with one 
embodiment of the present invention is shown in Fig. 7. 
[0078] The color gamut calculation section 1 60 calcu- 
lates and obtains the target color gamut, based on the 
target profile selected from the target profile storage 
55 section 162. The color gamut calculation section 160 
then calculates and obtains the displayable color gamut 
of the projector 20, based on the projector profile stored 
in the projector profile storage. section 1 64 and the tris- 
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timulus values detected by the colored-light sensor 60 
(stepS 12). 

[0079] The color gamut calculation section 1 60 com- 
pares the displayable color gamut and the target color 
gamut obtained by the calculations (step SI 2). 
[0080] First of all, if the displayable color gamut 
matches the target color gamut, in other words : if the 
situation is as shown in Fig. 2A (step 314), the matrix 
generation section 122 generates a conversion matrix 
that enables reproduction of the triangular mapping 
color gamut denoted by the solid line in Fig. 2A (step 
S16). 

[0081 ] If the displayable color gamut is larger than the 
target color gamut, in other words, if the situation is as 
shown in Fig. 2B (step S1 8), the matrix generation sec- 
tion 122 generates a conversion matrix that enables re- 
production of the triangular mapping color gamut denot- 
ed by the solid line in Fig. 2B (step S20). 
[0082] If the displayable color gamut is smaller than 
the target color gamut, in other words, if the situation is 
as shown in Fig. 3A. the matrix generation section 122 
generates a conversion matrix that enables reproduc- 
tion of a mapping color gamut that gives priority to either 
color gamut or hue reproduction, as shown in Fig. 4A or 
4B (step S24). 

[0083] If the situation is not one of the above three 
patterns (steps S14, S18. and S22), the displayable 
color gamut has a portion that overlaps the target color 
gamut and a portion that does not overlap, as shown in 
Fig. 3B. In that case ; the matrix generation section 122 
generates a conversion matrix that enables reproduc- 
tion of a mapping color gamut that gives priority to either 
color gamut or hue reproduction, as shown in Fig. 4A or 
4B (step S26). 

[0084] Note that all of the conversion matrices gener- 
ated by the individual matrix generation steps (steps 
Si 6, S20, S24 : and S26) are different. 
[0085] The matrix converter section 124 uses the con- 
version matrix generated by the matrix generation sec- 
tion 1 22 to perform color conversion (image information 
conversion) (step S28). More specifically, the matrix 
converter section 124 uses a 3x3 conversion matrix to 
convert the digital RGB signals (R2 : G2, B2) and outputs 
them as digital RGB signals (R3 : G3, B3). 
[0086] This can be expressed as an equation, as: (R3 : 
G3, B3) = M(R2, G2, B2), where M denotes the conver- 
sion matrix. 

[0087] The projector 20 uses the D/A converter sec- 
tion 180 to convert the thus-converted digital RGB sig- 
nals (R3, G3, B3) into analog form, then uses the con- 
verted analog RGB signals (R4, G4, B4) to display the 
actual presentation image (step S10). 
[0088] Thus this embodiment uses a conversion ma- 
trix to convert image information to enable the display 
of an image that conforms to the image characteristic 
selected by the user. 

[0089] This makes it possible to implement an image 
•display system -that-can display an image that conforms 



to the user's preferences. 

[0090] This embodiment also projects and displays an 
image from consideration of the visual environment, by 
using the colored-light sensor 60 to detect the visual en- 

5 vironment. 

[0091] It is therefore possible to display an image that 
is adjusted for the visual environment when the image 
is displayed, making it possible to absorb differences in 
display environment and thus display the same image 

10 regardless of the environment in which it is employed. 
This makes it possible to reproduce substantially the 
same colors within a short time, in a plurality of different 
locations. 

[0092] Furthermore, this embodiment makes it possi- 
15 ble to convert the image information more rapidly and 
reduce the amount of ded ; .-.ted storage required there- 
for, by using a conversion matrix for the image informa- 
tion conversion, not an LUT 

[0093] This embodiment also divides the relationship 
20 between displayable color gamut and target color gamut 
into four patterns during the generation of the conver- 
sion matrix, and generates a conversion matrix corre- 
sponding to the appropriate pattern. 
[0094] The relationship between the displayable color 
25 gamut and the target color gamut differs with the envi- 
ronment in which the projector 20 is used and the image 
characteristic selected by the user. For that reason, it is 
necessary to generate a conversion matrix that is ap- 
propriate for the relationship between the displayable 
30 color gamut and the target color gamut. 

[0095] With this embodiment of the invention, an ap- 
propriate conversion matrix can be generated by gen- 
erating a conversion matrix corresponding to any of four 
assumed patterns. 
35 [0096] Note that in the patterns shown in Figs. 2A and 
2B, it is possible that the target color gamut can be used 
substantially unchanged as the mapping color gamut, 
enabling faster conversion matrix generation than in the 
cases shown in Figs. 3A and 3B in which color gamut 
40 mapping is necessary. 

[0097] In addition, it is possible to reproduce an image 
more appropriately in the cases shown in Figs. 3A and 
3B in which color gamut mapping is necessary, by using 
a conversion matrix that emphases either hue reproduc- 
es jbility or color gamut reproducibility, in comparison with 
the use of a conversion matrix that emphases bright- 
ness or chroma. 

Hardware 

50 

[0098] Note that the hardware listed below could be 
applied as the portions described above. 
[0099] For example, the A/D converter section 110 
could be implemented by an A/D converter or the like, 
55 the D/A converter section 1 80 by a D/A converter or the 
like, the LA/ drive section 190 by a liquid crystal light 
valve driver or the like, the projector coior converter sec- 
tion 120 by image processing circuitry-or an ASIC, . and 



8 



15 



EP 1 205 902 A2 



16 



the color gamut calculation section 160 by a CPU and 
RAM, by way of example. Note that these portions could 
be implemented in a hardware fashion by circuitry, or 
they could be implemented in a software fashion by driv- 
ers and/or programs. 

[0100] In addition, the functions of the components 
shown in Fig. 5 could be implemented by reading out a 
program from an information storage medium 300. The 
information storage medium 300 could be a CD-ROM ; 
DVD-ROM, ROM,. RAM, or HDD : by way of example., 
and the method of reading the program therefrom could 
be a direct method or an indirect method. 
[0101] Instead of the information storage medium 
300, it is possible to implement the above described 
functions by downloading a program that implements 
those functions over a transfer path from a host device 
or the like. In other words, a program for implementing 
these functions could be embodied over carrier waves. 
[0102] The hardware described below could be em- 
ployed for the colored-light sensor 60. 
[01 03] For example, components such as a color filter 
and photodiode that selectively pass each stimulus val- 
ue, an A/D converterthat converts an analog signal from 
that photodiode into a digital signal, and an op-amp that 
amplifies that digital signal could be employed therefor. 
[01 04] Note the present invention has been described 
above by way of embodiments thereof, but the applica- 
tion of the present invention is not limited to the above 
embodiments. 

Modifications 

[0105] For example, an image characteristic such as 
RGB or sRGB image type could be employed as the tar- 
get profile, instead of the NTSC or similar image display 
method. 

[0106] Similarly, the visual environment detection 
means could be imaging means such as a CCD camera 
or a CMOS camera, instead of the colored-light sensor 
60. 

[01 07] Note that the abovementioned screen 1 0 could 
be of a reflective type or a transmissive type. 
[0108] Furthermore, the conversion matrix that was 
described above related to a single matrix, but the color 
conversion could also be done with a combination of a 
plurality of matrices. For example., the color conversion 
could be done with a combination of an inverse-conver- 
sion matrix corresponding to an output device and an 
environment compensation matrix that reflects environ- 
mental information. 

[0109] The present invention can also be applied to 
presentations in which images arc displayed by a dis- 
play means other than a projection-type image display 
device such as a projector. Apart from a liquid crystal 
projector, this display means could be a display device 
such as a cathode ray tube (CRT), a plasma display 
panel (PDP), a field emission device (FED), and electro- 
luminescence (EL) device, or a direct-view type of liquid 



crystal display device, or a projector using a digital mi- 
cromirror device (DMD), by way of example. Note the 
DMD is a tradename registered by Texas Instruments 
Inc.. of the US. In addition, the projector is not limited to 
5 a front-projection type; it could equally well be of a rear- 
projection type. 

[01 10] In addition to presentations., this invention is al- 
so effective when used in the display of images, such 
as those at meetings, during medical treatment, in the 

10 design and fashion fields, business activities, commer- 
cials, and education, as well as general-purpose images 
such as those in movies, TV, videos, and games. 
[0111] Note that the functions of the above-described 
projector image processor section 100 of the projector 

15 20 could be implemented by a simple image display de- 
vice (such as the projector 20 itself) or by distribution 
between a plurality of processing devices (such as dis- 
tributed processing between the projector 20 and a PC). 

20 

Claims 

1. An image display system which converts image in- 
formation used for displaying an image and displays 

25 the image, based on environmental information ob- 
tained by a visual environment detection means 
which detects visual environment in a display area 
in which the image is displayed; the image display 
system comprising: 

30 

matrix generation means which generates a 
conversion matrix used for conversion, based 
on the environmental information and an image 
characteristic selected by a user, so as to dis- 

35 play the image that is suited to the visual envi- 

ronment and the image characteristic; 
matrix conversion means which converts the 
image information, based on the generated 
conversion matrix; and 

40 image display means which displays the image 

based on the converted image information; 

wherein the image characteristic is based on 
at least one of an image display method and an im- 
45 age classification. 

2. The image display system as defined by claim 1 . 
further comprising: 

so a color gamut calculation means which calcu- 

lates a target color gamut based on the image 
characteristic and also calculates a displayable 
color gamut that is displayable by the image 
display means in the visual environment, based 

55 on the environmental information; 

wherein the matrix generation means gener- 
ates different conversion matrices according to a re- 
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lationship between the displayable color gamut and 
the target gamut, the relationship including a case 
where the displayable color gamut is larger than the 
target color gamut, a case where the displayable 
color gamut is smaller than the target color gamut : 
a case where the displayable color gamut matches 
the target color gamut, and a case where the dis- 
playable color gamut has a portion that overlaps the 
target color gamut and a portion that does not over- 
lap the target color gamut. 

3. The image display system as defined by claim 2, 

wherein the matrix generation means gener- 
ates the conversion matrix that emphases either 
hue reproducibility or color gamut reproducibility, 
when the displayable color gamut is smaller than 
the target color gamut or when the displayable color 
gamut has a portion that overlaps the target color 
gamut and a portion that does not overlap the target 
color gamut. 

4. The image display system as defined by any one of 
claims 1 to 3 : further comprising a projection -type 
display device having: 

the color gamut calculation means; 
the matrix generation means; 
the matrix conversion means; 
the image display means; and 
calibration image generation means which gen- 
erates a calibration image; 

wherein the image display means projects the 
generated calibration image for display on the dis- 
play area; and 

wherein the visual environment detection 
means detects the visual environment in the display 
area in which the calibration image is displayed. 

5. An image processing method which converts image 
information used for displaying an image, based on 
environmental information obtained by a visual en- 
vironment detection means which detects visual en- 
vironment in a display area in which the image is 
displayed., the image processing method compris- 
ing: 

a matrix generation step of generating a con- 
version matrix used for conversion, based on 
the environmental information and an image 
characteristic selected by a user, so as to dis- 
play the image that is suited to tho visual envi- 
ronment and the image characteristic: and 
a matrix conversion step of converting the im- 
age information., based on the generated con- 
version matrix. 

6. The image- processing method as-defined by claim 
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5, wherein: 

the matrix generation step includes a step of 
calculating a target color gamut based on the 
image characteristic and also calculating a dis- 
playable color gamut that is displayable by the 
image display means in the visual environment, 
based on the environmental information; and 
the matrix generation step generates different 
conversion matrices according to a relationship 
between the displayable color gamut and the 
target gamut, the relationship including a case 
where the displayable color gamut is larger 
than the target color gamut, a case where the 
displayable color gamut is smaller than the tar- 
get color gamut, a case where the displayable 
color gamut matches the target color gamut, 
and a case where the displayable color gamut 
has a portion that overlaps the largetcolor gam- 
ut and a portion that does not overlap the target 
color gamut. 

The image processing method as defined by claim 

6, 

wherein the matrix generation step generates 
the conversion matrix that emphases either hue re- 
producibility or color gamut reproducibility, when 
the displayable color gamut is smaller than the tar- 
get color gamut or when the displayable color gam- 
ut has a portion that overlaps the target color gamut 
and a portion that does not overlap the target color 
gamut. 

The image processing method as defined by any 
one of claims 5 to 7, further comprising the steps of: 

generating a calibration image, prior to com- 
pensation of the image information; 
displaying the generated calibration image in 
the display area; and 

detecting the visual environment within the dis- 
play area in which the calibration image is dis- 
played, and generating the environmental infor- 
mation. 

A computer-readable information storage medium 
which stores a program lor converting image infor- 
mation used for displaying an image, based on en- 
vironmental information obtained by a visual envi- 
ronment detection means which detects visual en- 
vironment in a display area in which the image is 
displayed, 

the program causing a computer to function 

as: 

matrix generation means which generates a 
conversion matrix used for conversion, based 
on the environmental information.and.aa image 
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characteristic selected by a user so as to dis- 
play the image that is suited to the visual envi- 
ronment and the image characteristic; and 
matrix conversion means which converts the 
image information based on the generated 5 
conversion matrix: and 

wherein the image characteristic is based on 
at least one of an image display method and an im- 
age classification. 10 

10. The information storage medium as defined by 
claim 9 : stores a program causing the computer to 
function as: 

15 

a color gamut calculation means which calcu- 
lates a target color gamut based on the image 
characteristic and also calculates a displayable 
color gamut that is displayable by the image 
display means in the visual environment, based 20 
on the environmental information; 

wherein the matrix generation means gener- 
ates different conversion matrices according to a re- 
lationship between the displayable color gamut and 25 
the target gamut, the relationship including a case 
where the displayable color gamut is larger than the 
target color gamut, a case where the displayable 
color gamut is smaller than the target color gamut, 
a case where the displayable color gamut matches 30 
the target color gamut, and a case where the dis- 
playable color gamut has a portion that overlaps the 
target color gamut and a portion that does not over- 
lap the target color gamut. 

35 

11. The information storage medium as defined by 
claim 10 : 

wherein the matrix generation means gener- 
ates the conversion matrix that emphases either 
hue reproducibility or color gamut reproducibility, -to 
when the displayable color gamut is smaller than 
the target color gamut or when the displayable color 
gamut has a portion that overlaps the target color 
gamut and a portion that does not overlap the target 
color gamut. 45 
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FIG. 2 A 
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FIG. 3A 
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FIG. 4 A 
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FIG. 6 
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(57) An image display system, image processing 
method, and program that make it possible to reproduce 
the colors of an image more rapidly in accordance with 
the user's preferences. The image display system in- 
cludes:- a color gamut calculation section 160 that cal- 
culates a target color gamut based on a target profile 
within a target profile storage section 162 that was se- 
lected by the user and also calculates a displayable 
color gamut based on a projector profile within a projec- 



tor profile storage section 164 and environmental infor- 
mation from a colored-light sensor 60 that detects the 
visual environment; a matrix generation section 1 22 that 
generates a conversion matrix according to the relation- 
ship between the target color gamut and the displayable 
color gamut; and a matrix converter section 1 24 that us- 
es the thus-generated conversion matrix to convert im- 
age information; whereby image information is convert- 
ed and image is displayed. 
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